1. Field
The present disclosure relates generally to image processing and, in particular, to enhancing images using super-resolution. Still more particularly, the present disclosure relates to a method and apparatus for enhancing video containing moving objects using super-resolution.
2. Background
Currently, many different algorithms are available for performing super-resolution. Super-resolution is a process in which images of the same scene are used to reconstruct an image with a higher resolution than the resolution of the images of the scene. As used herein, the term “resolution” is used to mean “spatial resolution”. Spatial resolution refers to the smallest possible feature that can be detected in an image. For example, smaller features can be detected in images having higher resolutions as compared to images having lower resolutions.
With super-resolution, multiple images of low resolution (LR) for a scene are used to reconstruct an image of high resolution (HR). Further, with super-resolution, the low resolution images used to construct the high resolution image cannot all be identical. Variation needs to be present between these low resolution images. For example, the low resolution images may be taken at different times, under different conditions, or both.
Typically, the low resolution images used to perform super-resolution are obtained from a sequence of images in the form of video. A sequence of images is a plurality of images ordered with respect to time. The different types of variation that may be present in these images include, for example, translational motion, rotational motion, different viewing angles, other types of motion, or any of these different types of variations in any combination. Translational motion between two images occurs when a current image of a scene is translated with respect to a previous image of the same scene parallel to the image plane. Rotational motion occurs when the current image is rotated with respect to the previous image. In some cases, variation is created when the current image is generated from a position closer to or further away from the scene as compared to the previous image.
With super-resolution, the low resolution images are first registered with respect to a selected reference coordinate system. Registration of these low resolution images includes aligning these images with respect to a reference coordinate system. For example, features in a first image may be aligned with the same features in a second image. However, when moving objects are present in the scene with respect to a background in the scene, registration of the low resolution images of the scene may not account for the moving objects. As a result, the high resolution image constructed from these low resolution images may be less accurate than desired.
Therefore, it would be advantageous to have a method and apparatus that takes into account at least some of the issues discussed above, as well as possibly other issues.